Pragmatic Multi-Agent Learning

Early models of procedural learning assumed actors were isolated, model-based thinkers. More recently, learning techniques have become more sophisticated as this assumption has been replaced with more realistic ones. To date, however, there has been no thorough investigation of multiple, heterogeneous, situated agents who learn from the pragmatics of their domain rather than from a model. This research focuses on this important problem and develops learning techniques that allow agents to improve their performance in a dynamic environment by learning from past run-time behavior. Humans provide a natural model of pragmatic agents situated in a multi-agent world. [1] argues that the development of distributed cooperative behavior in people is shaped by the accumulated cultural-historical knowledge of the community. Our learning techniques are motivated by this argument and use a structure called collective memory to store the accumulated procedural knowledge of a community of agents. Collective memory contains the breadth of knowledge the community acquires through interacting with each other and the world during the course of solving sequences of distinct problems. The cornerstone of collective memory is a cooperative procedures case-base that augments the agents’ first-order planner [2]; in other words, this work follows in the tradition of second-order planners, extending them into multi-agent domains. In our model of activity, each agent has her own point of view on how best to proceed, which often leads to uncoordinated and unproductive behavior. Furthermore, inefficient behavior would occur even if there was a consensus upon the best course of action to follow (perhaps legislated by a supervising agent or agreed to during community-wide communication) because of the community’s initial lack of knowledge about their uncertain domain. Through the use of collective memory, however, agents behave more efficiently over the course of solving a problem sequence for two reasons. First, individual agents develop a point of view based upon shared experiences; second, they learn procedures that capture regularities both in the task environment and in the patterns of cooperation for solving problems in task domain. That is, an agent remembers successful cooperative behavior in which she was involved, and uses it as a basis for future interactions.